Goto

Collaborating Authors

 White Sea


GENUINE: Graph Enhanced Multi-level Uncertainty Estimation for Large Language Models

Wang, Tuo, Kulkarni, Adithya, Cody, Tyler, Beling, Peter A., Yan, Yujun, Zhou, Dawei

arXiv.org Artificial Intelligence

Uncertainty estimation is essential for enhancing the reliability of Large Language Models (LLMs), particularly in high-stakes applications. Existing methods often overlook semantic dependencies, relying on token-level probability measures that fail to capture structural relationships within the generated text. We propose GENUINE: Graph ENhanced mUlti-level uncertaINty Estimation for Large Language Models, a structure-aware framework that leverages dependency parse trees and hierarchical graph pooling to refine uncertainty quantification. By incorporating supervised learning, GENUINE effectively models semantic and structural relationships, improving confidence assessments. Extensive experiments across NLP tasks show that GENUINE achieves up to 29% higher AUROC than semantic entropy-based approaches and reduces calibration errors by over 15%, demonstrating the effectiveness of graph-based uncertainty modeling. The code is available at https://github.com/ODYSSEYWT/GUQ.


CloudCast -- Total Cloud Cover Nowcasting with Machine Learning

Partio, Mikko, Hieta, Leila, Kokkonen, Anniina

arXiv.org Artificial Intelligence

Cloud cover plays a critical role in weather prediction and impacts several sectors, including agriculture, solar power generation, and aviation. Despite advancements in numerical weather prediction (NWP) models, forecasting total cloud cover remains challenging due to the small-scale nature of cloud formation processes. In this study, we introduce CloudCast, a convolutional neural network (CNN) based on the U-Net architecture, designed to predict total cloud cover (TCC) up to five hours ahead. Trained on five years of satellite data, CloudCast significantly outperforms traditional NWP models and optical flow methods. Compared to a reference NWP model, CloudCast achieves a 24% lower mean absolute error and reduces multi-category prediction errors by 46%. The model demonstrates strong performance, particularly in capturing the large-scale structure of cloud cover in the first few forecast hours, though later predictions are subject to blurring and underestimation of cloud formation. An ablation study identified the optimal input features and loss functions, with MAE-based models performing the best. CloudCast has been integrated into the Finnish Meteorological Institute's operational nowcasting system, where it improves cloud cover forecasts used by public and private sector clients. While CloudCast is limited by a relatively short skillful lead time of about three hours, future work aims to extend this through more complex network architectures and higher-resolution data. CloudCast code is available at https://github.com/fmidev/cloudcast.


Crossroads of Continents: Automated Artifact Extraction for Cultural Adaptation with Large Multimodal Models

Mukherjee, Anjishnu, Zhu, Ziwei, Anastasopoulos, Antonios

arXiv.org Artificial Intelligence

In this work, we present a comprehensive three-phase study to examine (1) the effectiveness of large multimodal models (LMMs) in recognizing cultural contexts; (2) the accuracy of their representations of diverse cultures; and (3) their ability to adapt content across cultural boundaries. We first introduce Dalle Street, a large-scale dataset generated by DALL-E 3 and validated by humans, containing 9,935 images of 67 countries and 10 concept classes. We reveal disparities in cultural understanding at the sub-region level with both open-weight (LLaVA) and closed-source (GPT-4V) models on Dalle Street and other existing benchmarks. Next, we assess models' deeper culture understanding by an artifact extraction task and identify over 18,000 artifacts associated with different countries. Finally, we propose a highly composable pipeline, CultureAdapt, to adapt images from culture to culture. Our findings reveal a nuanced picture of the cultural competence of LMMs, highlighting the need to develop culture-aware systems. Dataset and code are available at https://github.com/iamshnoo/crossroads


SOMson -- Sonification of Multidimensional Data in Kohonen Maps

Linke, Simon, Ziemer, Tim

arXiv.org Artificial Intelligence

Kohonen Maps, aka. Self-organizing maps (SOMs) are neural networks that visualize a high-dimensional feature space on a low-dimensional map. While SOMs are an excellent tool for data examination and exploration, they inherently cause a loss of detail. Visualizations of the underlying data do not integrate well and, therefore, fail to provide an overall picture. Consequently, we suggest SOMson, an interactive sonification of the underlying data, as a data augmentation technique. The sonification increases the amount of information provided simultaneously by the SOM. Instead of a user study, we present an interactive online example, so readers can explore SOMson themselves. Its strengths, weaknesses, and prospects are discussed.


A Parallel Workflow for Polar Sea-Ice Classification using Auto-labeling of Sentinel-2 Imagery

Iqrah, Jurdana Masuma, Wang, Wei, Xie, Hongjie, Prasad, Sushil

arXiv.org Artificial Intelligence

The observation of the advancing and retreating pattern of polar sea ice cover stands as a vital indicator of global warming. This research aims to develop a robust, effective, and scalable system for classifying polar sea ice as thick/snow-covered, young/thin, or open water using Sentinel-2 (S2) images. Since the S2 satellite is actively capturing high-resolution imagery over the earth's surface, there are lots of images that need to be classified. One major obstacle is the absence of labeled S2 training data (images) to act as the ground truth. We demonstrate a scalable and accurate method for segmenting and automatically labeling S2 images using carefully determined color thresholds. We employ a parallel workflow using PySpark to scale and achieve 9-fold data loading and 16-fold map-reduce speedup on auto-labeling S2 images based on thin cloud and shadow-filtered color-based segmentation to generate label data. The auto-labeled data generated from this process are then employed to train a U-Net machine learning model, resulting in good classification accuracy. As training the U-Net classification model is computationally heavy and time-consuming, we distribute the U-Net model training to scale it over 8 GPUs using the Horovod framework over a DGX cluster with a 7.21x speedup without affecting the accuracy of the model. Using the Antarctic's Ross Sea region as an example, the U-Net model trained on auto-labeled data achieves a classification accuracy of 98.97% for auto-labeled training datasets when the thin clouds and shadows from the S2 images are filtered out.


Adversarial Attacks on Image Classification Models: FGSM and Patch Attacks and their Impact

Sen, Jaydip, Dasgupta, Subhasis

arXiv.org Artificial Intelligence

This chapter introduces the concept of adversarial attacks on image classification models built on convolutional neural networks (CNN). CNNs are very popular deep-learning models which are used in image classification tasks. However, very powerful and pre-trained CNN models working very accurately on image datasets for image classification tasks may perform disastrously when the networks are under adversarial attacks. In this work, two very well-known adversarial attacks are discussed and their impact on the performance of image classifiers is analyzed. These two adversarial attacks are the fast gradient sign method (FGSM) and adversarial patch attack. These attacks are launched on three powerful pre-trained image classifier architectures, ResNet-34, GoogleNet, and DenseNet-161. The classification accuracy of the models in the absence and presence of the two attacks are computed on images from the publicly accessible ImageNet dataset. The results are analyzed to evaluate the impact of the attacks on the image classification task.


UW-CVGAN: UnderWater Image Enhancement with Capsules Vectors Quantization

Pucci, Rita, Micheloni, Christian, Martinel, Niki

arXiv.org Artificial Intelligence

The degradation in the underwater images is due to wavelength-dependent light attenuation, scattering, and to the diversity of the water types in which they are captured. Deep neural networks take a step in this field, providing autonomous models able to achieve the enhancement of underwater images. We introduce Underwater Capsules Vectors GAN UWCVGAN based on the discrete features quantization paradigm from VQGAN for this task. The proposed UWCVGAN combines an encoding network, which compresses the image into its latent representation, with a decoding network, able to reconstruct the enhancement of the image from the only latent representation. In contrast with VQGAN, UWCVGAN achieves feature quantization by exploiting the clusterization ability of capsule layer, making the model completely trainable and easier to manage. The model obtains enhanced underwater images with high quality and fine details. Moreover, the trained encoder is independent of the decoder giving the possibility to be embedded onto the collector as compressing algorithm to reduce the memory space required for the images, of factor $3\times$. \myUWCVGAN{ }is validated with quantitative and qualitative analysis on benchmark datasets, and we present metrics results compared with the state of the art.


Good News Roundup: the OSINT-inspired Geek Edition

#artificialintelligence

In this week's geeked-out edition of the Good News Roundup, Ukraine's jaw-dropping battlefield victories with HIMARS are documented using OSINT, South Africa implements AI technology to track dangerous locust swarms, biologists and naturalists overwhelmingly agree that gay sex is normal throughout the animal kingdom, and BirdNet proves reliable at crowdsourcing the task of identifying wild birds by their songs. In wholesome news for sci fi/space fantasy fans everywhere, Ukraine's president Zelensky continues attending technology trade shows through holograms in which he promises that Ukraine will defeat the Empire. Ukrainians are also using 3d imaging technology to preserve the cultural heritage of their country from looters and bombs, storing their data in a digital archive that will support restoration work when the invaders have been defeated. And in good news for new Ukrainian parents, the non-profit Embrace Global is making headlines for using innovative technology to provide incubators for babies in Ukraine at a tiny fraction of their usual cost. You can see their TED talk by entrepreneur Jane Chen here.


Function Contrastive Learning of Transferable Representations

Gondal, Muhammad Waleed, Joshi, Shruti, Rahaman, Nasim, Bauer, Stefan, Wüthrich, Manuel, Schölkopf, Bernhard

arXiv.org Machine Learning

Few-shot-learning seeks to find models that are capable of fast-adaptation to novel tasks. Unlike typical few-shot learning algorithms, we propose a contrastive learning method which is not trained to solve a set of tasks, but rather attempts to find a good representation of the underlying data-generating processes (\emph{functions}). This allows for finding representations which are useful for an entire series of tasks sharing the same function. In particular, our training scheme is driven by the self-supervision signal indicating whether two sets of samples stem from the same underlying function. Our experiments on a number of synthetic and real-world datasets show that the representations we obtain can outperform strong baselines in terms of downstream performance and noise robustness, even when these baselines are trained in an end-to-end manner.


The Unseen

The New Yorker

Once a year, when Slava Epstein was growing up in Moscow, his mother took him to the Exhibition of the Achievements of the National Economy, a showcase for the wonders of Soviet life. The expo featured many things--from industrial harvesters to Uzbek wine--but Epstein, who began going in the nineteen-sixties, when he was eight or nine, was interested primarily in one: the Cosmos Pavilion, a building the size of a hangar, with a ceiling shaped like a giant inverted parabola. Space fever was running high in the city. Since 1961, when Yuri Gagarin orbited the globe, unmanned vessels had been launched toward Mars and Venus. Beside the expo's entrance, the towering Monument to the Conquerors of Space depicted a probe swooping up to the heavens. The Pavilion displayed futuristic technology--Vostok rockets and Soyuz orbiters--but Epstein was less interested in the glories of advanced thruster design than in the glories of space. He wanted to devote himself to astronomy. When a textbook that he found on the topic began with algebraic formulas, he prodded his older brother to explain them. During high school, he enrolled in classes in physics and math at Moscow State University. His parents disapproved of his desired career: because he is half Jewish, Epstein would face harsh Soviet quotas limiting Jews in the study of physics, a field deemed relevant to national security. But after his first lecture the professor invited him for a walk, and affirmed what they had been saying all along. "Don't do it," he warned. Soviet Russia may have been a fatalist's paradise, but from a young age Epstein felt that he was hardwired for optimism. He convinced himself that what is truly important in science is the ability to connect ideas, no matter the field, and so he took up biology. Rather than telescopes, he would use microscopes, which he began taking with him on trips to the White Sea, near the Arctic Circle, to study protozoa along the shore--research that could be conducted with minimal state interference. Over time, he grew interested in even smaller, more ancient forms of life: bacteria. Studying microbes inevitably causes a reordering of one's perceptions: for more than two billion years, they were the only life on this planet, and they remain in many ways its dominant life form. To a remarkable extent, the microbial cosmos was less explored than the actual cosmos: precisely how the organisms evolve, replicate, fight, and communicate remains unclear. Nearly all of microbiology, Epstein eventually learned, was built on the study of a tiny fraction of microbial life, perhaps less than one per cent, because most bacteria could not be grown in a laboratory culture, the primary means of analyzing them. By the time he matured as a scientist, many researchers had given up trying to cultivate new species, writing off the majority as "dark matter"--a term used in astronomy for an inscrutable substance that may make up most of the universe but cannot be seen.